{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    ".. _`userguide/connector`:\n",
    "\n",
    "Connector\n",
    "=========\n",
    "\n",
    "DataPrep.Connector aims to simplify data collection from Web APIs and databases by providing a standard set of operations. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Web APIS\n",
    "Connector wraps-up complex API calls into a set of easy-to-use Python functions. \n",
    "By using Connector, you can skip the complex API configuration process and rapidly query different Web APIs in few steps, enabling you to execute the analysis workflow you are familiar with in a direct way.\n",
    "\n",
    "Watch our introduction in PyGlobal Conference [here](https://www.youtube.com/watch?v=56qu-0Ka-dA/).\n",
    "\n",
    "Connector offers essential features to facilitate the process of collecting data, for example:\n",
    "\n",
    "* Concurrency: Collect data from websites, in parallel, in a fast way!\n",
    "\n",
    "* Pagination: Retrieve more rows of a particular query without getting into unnecessary detail about pagination schemes!\n",
    "\n",
    "* Authorization: Access more Web APIs quickly! Even the ones that implement authorization!\n",
    "\n",
    "The user guide first presents a case study for [dblp](https://dblp.org/) as an example for the process overview and provides a detailed explanation of the functionalities in the following sections.\n",
    "\n",
    "## DBs\n",
    "Connector wraps on [connectorx](https://github.com/sfu-db/connector-x) to allow user fetch data from databases through SQL query. The result of the query will be stored into a Python dataframe.\n",
    "\n",
    "#### Supported databases:\n",
    "* Postgres\n",
    "* Mysql\n",
    "* Sqlite\n",
    "* SQL Server\n",
    "* Oracle\n",
    "* Redshift (through postgres protocol)\n",
    "* Clickhouse (through mysql protocol)\n",
    "\n",
    "#### Supported dataframes:\n",
    "* Pandas\n",
    "* Arrow\n",
    "* Dask\n",
    "* Modin\n",
    "* Polars\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbsphinx-toctree": {
     "maxdepth": 2,
     "titlesonly": true
    },
    "tags": [
     "nbsphinx-toctree"
    ]
   },
   "source": [
    "## Section Contents\n",
    "\n",
    " * [Case study: dblp API](dblp.ipynb)\n",
    " * [Configuration files: usage and composition](config.ipynb)\n",
    " * [connect(): establish the connection and load the configuration files](connect.ipynb)\n",
    " * [query(): fetch data into DataFrames via APIs](query.ipynb)\n",
    " * [info(): get information of a website](info.ipynb)\n",
    " * [Authorization schemes supported](authorization.ipynb)\n",
    " * [Auto-pagination](pagination.ipynb)\n",
    " * [read_sql(): fetch data into DataFrames via SQL query](sql.ipynb)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Edit Metadata",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
